Update ingestion helper to use cloud run services#1999
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request migrates the import automation infrastructure from Cloud Functions to Cloud Run services, containerizing the ingestion and import helpers and consolidating aggregation logic. Several critical issues were identified: the aggregation_utils.py file is missing from the PR, the SPANNER_GRAPH_DATABASE_ID environment variable is not configured for the new service, and removing DDL management from Terraform will break database initialization. Furthermore, the build process is brittle due to remote schema fetching, and a stale default URL persists in the update script.
gmechali
left a comment
gmechali
left a comment
There was a problem hiding this comment.
Just one change needed for the schema download on the dockerfile but otherwise LGTM.
Pls share this with Sandeep as well, so he doesn't waste time starting from an old branch!
2 participants
Updated ingestion workflow, cloud build, and Terraform scripts to use cloud run services instead of cloud function for helpers.
Moved aggregation helper logic into ingestion helper to consolidate under a single docker image. We can always deploy them as independent services if required.